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Preface 



The world in which intelligence analysts work has changed 
dramatically over the 67 years since the beginning of the Second 
World War. Adversaries have shifted from large armies arrayed on 
battlefields to individuals lurking in the shadows or in plain sight. 
Further, plagues and pandemics, as well as floods and famines, pose 
threats not only to national stability but even to human existence. To 
paraphrase a Chinese curse, we certainly live in interesting times. 

Our times demand fresh, critical reasoning on the part of those 
tasked to assess and warn about threats as well as those tasked to act 
on those threats. Education in the bases and practices of intelligence 
foraging and sensemaking - often called intelligence collection and 
analysis - is a means by which this can be accomplished. Indeed, the 
Intelligence Reform and Terrorism Prevention Act of 2004 legislates 
improved education for intelligence analysis. But, that education is 
not specifically defined. This volume provides a framework for one 
area of the act’s educational requirement: improving how analysts 
think - and by extension, how policymakers act. It asserts that 
people who are skilled critical thinkers are better able to cope with 
interesting times than those who are not. 

The model for thinking developed here also provides specific 
tools for coping with accelerating disruptive technologies. Such 
technologies routinely appear in the hands of adversaries. They 
also offer intelligence professionals capabilities to counter adversaries 
in novel ways. The key is knowing which technologies are truly 
disruptive in advance, which pose threats, and which can be 
harnessed to mitigate threats. Critical thinking - as it is here defined 
and developed - provides part of the solution as it encourages 
careful consideration of the available evidence, close examination of 
presuppositions and assumptions, review of the alternate implications 
of decisions, and finally, discussion of alternative solutions and 
possibilities. In short, it equips intelligence professionals with an 
essential tool for their work. 
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Foreword 



Mark M. Lowenthal 

Former Assistant Director of Central Intelligence 
for Analysis and Production 
President, Intelligence & Security Academy, LLC 



Some years ago, when I worked at the Congressional Research 
Service, my colleague next door had a sign in his office: “Thinking 
is hard work. That’s why so few people do it.” Indeed. As sentient 
beings we think all the time. Our waking, conscious hours are full 
of thoughts: what to wear; what to eat; how to respond to someone. 
These all take thought. One might venture to say that we think 
almost unconsciously through most of our day. 

And then there is that other thinking: Thinking about the 
conundrums that we face, the alternatives and choices we have to 
make on larger issues, the dilemmas we wish to resolve. This is hard 
work. And if the work you do is largely intellectual in nature, then 
you are thinking that much more often about these more difficult 
problems. This is not to suggest that those who work with their 
hands or in crafts or industry do not think. Of course they do. But 
these people, in the end, have something physical to show for their 
thought and its execution. Intellectuals only have their thoughts to 
show, presumably expressed for others in some way. 

Intelligence officers are engaged in an intellectual pursuit. They 
are trying to solve puzzles, resolve uncertainties, discover the nature 
and meaning of things that others would keep secret. They must have 
an entire intellectual apparatus to help them identify the problem, 
assess the parts they know and the parts they do not, come up with 
an explanation of what is going on and then express it in a way that 
others - including an audience not steeped in their own techniques 
— can understand. 
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As one would expect, the Intelligence Community has spent a 
fair amount of time thinking about how it does analysis. But most 
of this has been spent on techniques, presentation and outcomes; 
much less has been spent on what happens intellectually while you are 
thinking. That, indeed, is what critical thinking is about: the ability 
to step out of one’s thoughts, if you will, to examine them and the 
process that brought them about even while you are thinking them. 
It does not require some higher level of cosmic consciousness or a 
split personality. It does require training and thought! So now we 
are thinking about our thinking while we are thinking — a mental 
triple play, if you will. This is no mean feat and it is a very crucial 
skill for intelligence analysts. 

John McLaughlin, a former Deputy Director of Central 
Intelligence, has observed that one of the major perceptions that 
separates intelligence officers from the policy makers is that policy 
makers tend to be optimists, believing they can get things done, 
while intelligence officers tend to be skeptics, knowing that little 
is known with certainty. Like everyone else, however, intelligence 
analysts can be adamant about their conclusions, understanding the 
path they took to reach them, and well-satisfied that they have done 
the right work and come to the right end. That is where critical 
thinking comes in. The ability to examine how you came to your 
conclusion is an important adjunct to your other intellectual work. 
You may be certain or you may not - which is also acceptable. But 
you owe it to yourself and to your readers to have examined how 
you got there. 

This is not as easy as it sounds for intelligence analysts for the 
simple reason that they are usually very pressed for time. There 
are not a lot of lulls in the analyst's work week, which makes it very 
difficult to be introspective. This means that critical thinking ought 
to be something that is ingrained in analysts as part of their training, 
so that it becomes reflexive. But to do that, one must first understand 
critical thinking - what it is, how to do it, how to teach or learn it. 
That is the outstanding value of this monograph by David Moore. 
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The Intelligence Community is filled with people happily 
working away under a cloak of anonymity, satisfied in making their 
contribution to part of a larger activity about which very few beyond 
their immediate circle of co-workers will know. David Moore has 
long been one of these contributors. In addition to his duties as an 
intelligence officer at the National Security Agency, David Moore 
has devoted many fruitful hours to the intellectual underpinnings of 
intelligence, especially to what makes analysts and what makes better 
analysts. His work on analytic core competencies, written with his 
colleague Lisa Krizan, is a fascinating dissection of the intellectual 
attributes that make up an intelligence analyst. It was also a very 
influential work, becoming a prime motivator in the creation of 
the Analytic Resources Catalog created under Director of Central 
Intelligence George Tenet and now much touted by Director of 
National Intelligence John Negroponte. 

This new monograph on critical thinking will serve a similar 
purpose. Much has been written in the last five years about intelligence 
and about the errors that it can make. Some of this has been 
diagnostic; much of it has been blithely condemnatory. Prescriptions 
about connecting dots better (an absurd metaphor), about the need to 
share information better and alarms about groupthink do not get us 
closer to improvements in the art, craft and profession of intelligence. 
New ways of thinking about our profession, new ways of doing our 
work and new ways of assessing it will be useful. Critical thinking 
needs to be at the top of this list of desiderata. 

I would like to believe that we are on the verge of a renaissance in 
intelligence studies, and more specifically in the intellectual basis of 
intelligence analysis. There are many straws in the wind: discussions 
about a national intelligence university and a lessons learned center, 
more schools offering serious courses about intelligence and books 
like this. Critical thinking is and should be critical for all analysts 
and they will profit from this book. 
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Commentary: 



Jeffrey R. Cooper 
SAIC Technical Fellow 

Science Applications International Corporation 



David Moore has written an elegant and largely persuasive 
argument for the Intelligence Community to move forthrightly 
and adopt critical thinking in order to improve the quality both of 
its analytic processes and its intelligence judgments. Since I have 
recently written about the need for the Community to identify and 
cure deep-seated and systemic “analytic pathologies,” I am obviously 
sympathetic to the general case for incorporating more rigorous 
analytic processes — among which “critical thinking” is one attractive 
approach . 1 Along with Moore, I also believe strongly that the traits 
often associated with critical thinking need to become fundamental 
characteristics of every analyst and to be practiced consistently in 
their work habits. Moore discusses these aspects at some length and 
highlights their value to making sounder judgments. 

In doing so, he makes a compelling case that critical thinking 
can help analysts reshape their methods in ways that help to avoid 
damaging errors and enhance judgment. He and I agree with Rieber 
and Thomason that we should validate the potential improvement 
before widespread deployment . 2 The need for validation raises 
some related points concerning the criteria for evaluation and 
measurement. As Philip Tetlock has noted, on the one hand, one 
can do evaluation by assessing whether a process (assumed to be 
good) has been followed; one the other hand, one can also assess 



1 Jeffrey R. Cooper, “Curing Analytic Pathologies: Pathways to Improved 
Intelligence Analysis" (Washington, DC: Central Intelligence Agency, Center for 
the Study of Intelligence, 2005). Cited hereafter as Cooper, Pathologies. 

2 Steven Rieber and Neil Thomason, “Toward Improving Intelligence 
Analysis: Creation of a National Institute for Analytic Methods,” Studies in Intelligence 
49, no. 4 (Winter 2006), 71. 
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quality by looking at the products themselves. 3 However, following 
a good process does not guarantee good products, nor does a good 
judgment necessarily validate the quality of the analysis or the analyst. 
Both approaches have flaws as methods of validation. 

I urge the leadership of the Intelligence Community to place 
far more emphasis on structured analytic methods. In my view, 
the transformation of the intelligence enterprise demands a 
more curious, more agile, and more deeply thoughtful cadre of 
intelligence analysts — but it should also require the same traits 
among its intelligence organizations and the intelligence enterprise 
as a whole. Moore notes that “Investment in critical thinking as 
part of the analysis process minimizes the likelihood of specific 
failures” (page 81). However, from my perspective, critical thinking 
(and other structured methods) are more important for changing the 
organization’s overall approach to analysis, rather than in improving 
specific judgments or preventing particular failures. 

I believe such methods are crucial in preventing systemic analytic 
pathologies from developing, exactly because the Community lacks 
many of the desirable self-corrective mechanisms found in science. 
Second, while Moore focuses on the role of critical thinking in 
improving an individual analyst’s ability to make good judgments, 
my view emphasizes the importance of more rigorous processes for 
the organization as a whole if it is to improve its capacity to meet 
user needs. Indeed, given my emphasis on the systemic nature of 
the pathologies that afflict intelligence analysis, structured analytic 
methods become a first line of defense in preventing networks of 
errors — they are like “ripstops” that keep problems from propagating 
into wider “error-inducing systems,” in Perrow’s terms. The quality 
of “mindfulness” and a more self-reflective process are essential if 
the intelligence organizations are to acquire some of the desirable 
characteristics of high-reliability organizations. While critical thinking 
can clearly assist individual analysts, and these “memes” can be 
spread through viral dissemination, I would place far more emphasis 
on fomenting social and organizational changes as remedies. 

3 Philip E. Tetlock, Expert Political Judgment: How Good Is It? How Can We 

Know? (Princeton, NJ: Princeton University Press, 2005). 
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Finally, exactly because Moore focuses so tightly on the value of 
critical thinking to improve how individual analysts reason, he largely 
ignores the important role such habits should, and could, play in the 
“social construction” of intelligence analysis and its communication to users 
of that intelligence. The use of more structured techniques “to present 
more effective assessments” should go beyond simply convincing 
policymakers “to question their own assumptions on the issues” (page 
80). By restructuring both the analytic process and the modalities of 
interactions with policy users, critical thinking techniques can draw 
policy users into “co-creating” important judgments and, by doing 
so, get them to adopt some of the important attributes of critical 
thinking. This internalization would be more powerful than hoping 
the exquisite brilliance of the improved analytic judgments might 
make them more palatable. 

These minor quibbles aside, David Moore has highlighted for 
the Intelligence Community and its users some tangible methods 
for improving intelligence analysis and addressing systemic analytic 
pathologies. 
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Commentary 



Francis J. Hughes 

Professor, National Defense Intelligence College 



The truly difficult problems of the information age are not 
technological; rather, they concern ourselves - what it is to 
think, to reason, and to engage in conversation [using] . . . 
new analytic techniques, new conceptual tools with which 
to analyze and understand the workings of the human 
mind. 

— Keith Devlin, Goodbye Descartes: The End of Logic 
and the Search for a New Cosmology of the Mind, 1997 

David Moore has undertaken a valuable initiative on behalf 
of the Intelligence Community by producing this monograph on 
critical thinking. In doing so, he has addressed the concerns of 
Keith Devlin and many other scholarly writers on the subject of 
human reasoning. By contextualizing the concept and practices 
of critical thinking within intelligence analysis, he has set forth a 
much-needed cognitive linkage between intelligence analysis and 
the human thought process, which, as he states, remains otherwise 
“poorly understood.” 

Mr. Moore has developed a methods-of-thinking course, outlined 
in the Appendix to this paper, which should guide the collection 
of evidence, the reasoning from evidence to argument, and the 
application of objective decision-makingjudgment. Critical thinking 
in intelligence depends primarily on a conscious application of 
suitable habits of thought, and the course certainly promises to 
advance the education of intelligence professionals. 
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Commentary 



Gregory F. Treverton 
RAND Corporation 

Former Assistant Director of Central Intelligence for 
Analysis and Production 

and Vice-Chairman, National Intelligence Council 



David Moore has added his powerful voice to those calling for 
America’s intelligence analysts to be more self-conscious about their 
methods and more venturesome in applying more formal methods. 
His starting point is critical thinking, but his strong case embraces 
a range of techniques from Paul and Elder’s checklist, to analysis 
of competing hypotheses (ACH), to Factions analysis, which is really 
a way to be more systematic in aggregating subjective judgments, 
to techniques such as those developed by the Novel Intelligence 
from Massive Data (NIMD) project. These help analysts search for 
patterns and test hypotheses. Moore’s discussion pays particular 
attention to critical thinking about evidence - that is perhaps natural 
given that the National Security Agency is his home agency, but it 
is also a welcome emphasis. 

My own explorations of the analytic agencies over the last few 
years confirm my own experience managing the National Intelligence 
Council (NIC): In general, U.S. intelligence analysis has been neither 
very self-conscious about its methods, nor made much use of machines 
and formal analytic tools. That is Moore’s point of departure, and it 
is a case made graphically by Rob Johnston, on whose work Moore 
draws. The state of affairs Moore and others describe is changing, 
and his work should contribute to accelerating that change, in a 
variety of experiments and pilot projects. 

Moore’s paper raises at least two intriguing questions at its edges. 
He rightly takes issue with Malcolm Gladwell’s book about the 
power of unconscious thought or “deliberation without attention.” 
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That said, clever defense lawyers often discredit ballistic experts by 
asking them to be explicit about how they reached their conclusions. 
Likewise, chess masters can be reduced to middle-weights by having 
to explain the logic of their moves. In both cases, the decisions result 
from patterns as much sensed as seen, the result of thousands of 
previous cases. I sometimes worried at the NIC that the more we 
required analysts to be explicit about their methods, the more we 
risked turning them into middle-weights. 

Finding ways to incorporate not just expertise, but also hunch 
and sense into intelligence analysis is a challenge that lies ahead. The 
Defense Advanced Research Projects Agency’s ill-fated experiment 
a few years ago seeking to establish a terrorism futures market was 
testimony to how hard the challenge will be. When the idea became 
public, every politician - right, left and center - quickly denounced 
it as borderline immoral. Over the ensuing days, however, a wide 
range of analysis commented on what a good idea it might have 
been when information is in short supply and there is a premium 
on sensing patterns before they become facts. 

The second issue is consumers of intelligence, who figure only at 
the margins of Moore’s discussion. We can only hope he is right that 
better analysis will find, and perhaps help create, a better audience. 
But the unboundedness of the terrorist threat we now face means 
that intelligence and policy no longer share a “story” comparable 
to that about state threats, like the Soviet Union. In principle, that 
would suggest that policy should accord more time and attention 
to intelligence, but that is not likely to happen in Washington’s 
whirligig. 

Thus, intelligence will need to be as creative in finding new ways 
to work and communicate with policy officials as it is in improving 
its analytic techniques. In the process, it will need to rethink what 
it regards as its product. I came to think at the NIC that our product 
was not really National Intelligence Estimates but, rather, National 
Intelligence Officers - people, not paper, and people in a position to 
come to know their policy counterparts and to make their expertise 
available informally, in meetings or at lunch. 
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Abstract 



Analysts and analysts alone create intelligence. Although 
technological marvels assist analysts by cataloguing and presenting 
data, information and evidence in new ways, they do not do analysis. 
To be most effective, analysts need an overarching, reflective 
framework to add structured reasoning to sound, intuitive thinking. 
“Critical thinking” provides such a framework and goes further, 
positively influencing the entire intelligence analysis process. Analysts 
who adopt critical thinking stand to improve their analyses. This 
paper defines critical thinking in the context of intelligence analysis, 
explains how it influences the entire intelligence process, explores 
how it toughens the art of intelligence analysis, suggests how it may 
be taught, and deduces how analysts can be persuaded to adopt this 
habit. 
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Definitions 



Thinking - or reasoning - involves objectively connecting present 
beliefs with evidence in order to believe something else. 



Critical Thinking is a deliberate meta-cognitive (thinking about 
thinking) and cognitive (thinking) act whereby a person reflects on the 
quality of the reasoning process simultaneously while reasoning to 
a conclusion. The thinker has two equally important goals: coming 
to a solution and improving the way she or he reasons. 



Intelligence is a specialized form of knowledge, an activity, 
and an organization. As knowledge, intelligence informs leaders, 
uniquely aiding their judgment and decision-making. As an activity, 
it is the means by which data and information are collected, their 
relevance to an issue established, interpreted to determine likely 
outcomes, and disseminated to individuals and organizations who 
can make use of it, otherwise known as “consumers of intelligence.” 
An intelligence organization directs and manages these activities to 
create such knowledge as effectively as possible. 
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